Recurrent neural network models (CovRNN) for predicting outcomes of patients with COVID-19 on admission to hospital: model development and validation using electronic health record data.

BACKGROUND: Predicting outcomes of patients with COVID-19 at an early stage is crucial for optimised clinical care and resource management, especially during a pandemic. Although multiple machine learning models have been proposed to address this issue, because of their requirements for extensive data preprocessing and feature engineering, they have not been validated or implemented outside of their original study site. Therefore, we aimed to develop accurate and transferrable predictive models of outcomes on hospital admission for patients with COVID-19. METHODS: In this study, we developed recurrent neural network-based models (CovRNN) to predict the outcomes of patients with COVID-19 by use of available electronic health record data on admission to hospital, without the need for specific feature selection or missing data imputation. CovRNN was designed to predict three outcomes: in-hospital mortality, need for mechanical ventilation, and prolonged hospital stay (>7 days). For in-hospital mortality and mechanical ventilation, CovRNN produced time-to-event risk scores (survival prediction; evaluated by the concordance index) and all-time risk scores (binary prediction; area under the receiver operating characteristic curve [AUROC] was the main metric); we only trained a binary classification model for prolonged hospital stay. For binary classification tasks, we compared CovRNN against traditional machine learning algorithms: logistic regression and light gradient boost machine. Our models were trained and validated on the heterogeneous, deidentified data of 247 960 patients with COVID-19 from 87 US health-care systems derived from the Cerner Real-World COVID-19 Q3 Dataset up to September 2020. We held out the data of 4175 patients from two hospitals for external validation. The remaining 243 785 patients from the 85 health systems were grouped into training (n=170 626), validation (n=24 378), and multi-hospital test (n=48 781) sets. Model performance was evaluated in the multi-hospital test set. The transferability of CovRNN was externally validated by use of deidentified data from 36 140 patients derived from the US-based Optum deidentified COVID-19 electronic health record dataset (version 1015; from January, 2007, to Oct 15, 2020). Exact dates of data extraction were masked by the databases to ensure patient data safety. FINDINGS: CovRNN binary models achieved AUROCs of 93·0% (95% CI 92·6-93·4) for the prediction of in-hospital mortality, 92·9% (92·6-93·2) for the prediction of mechanical ventilation, and 86·5% (86·2-86·9) for the prediction of a prolonged hospital stay, outperforming light gradient boost machine and logistic regression algorithms. External validation confirmed AUROCs in similar ranges (91·3-97·0% for in-hospital mortality prediction, 91·5-96·0% for the prediction of mechanical ventilation, and 81·0-88·3% for the prediction of prolonged hospital stay). For survival prediction, CovRNN achieved a concordance index of 86·0% (95% CI 85·1-86·9) for in-hospital mortality and 92·6% (92·2-93·0) for mechanical ventilation. INTERPRETATION: Trained on a large, heterogeneous, real-world dataset, our CovRNN models showed high prediction accuracy and transferability through consistently good performances on multiple external datasets. Our results show the feasibility of a COVID-19 predictive model that delivers high accuracy without the need for complex feature engineering. FUNDING: Cancer Prevention and Research Institute of Texas.

DLMM as a lossless one-shot algorithm for collaborative multi-site distributed linear mixed models.

Linear mixed models are commonly used in healthcare-based association analyses for analyzing multi-site data with heterogeneous site-specific random effects. Due to regulations for protecting patients' privacy, sensitive individual patient data (IPD) typically cannot be shared across sites. We propose an algorithm for fitting distributed linear mixed models (DLMMs) without sharing IPD across sites. This algorithm achieves results identical to those achieved using pooled IPD from multiple sites (i.e., the same effect size and standard error estimates), hence demonstrating the lossless property. The algorithm requires each site to contribute minimal aggregated data in only one round of communication. We demonstrate the lossless property of the proposed DLMM algorithm by investigating the associations between demographic and clinical characteristics and length of hospital stay in COVID-19 patients using administrative claims from the UnitedHealth Group Clinical Discovery Database. We extend this association study by incorporating 120,609 COVID-19 patients from 11 collaborative data sources worldwide.

Short term visual and structural outcomes of anti-vascular endothelial growth factor (anti-VEGF) treatment delay during the first COVID-19 wave: a pilot study

Regularly scheduled intravitreal anti-vascular endothelial growth factor (anti-VEGF) injections are essential to maintaining and/or improving many ocular conditions including: neovascular age-related macular degeneration (nAMD), diabetic retinopathy, and retinal vein occlusions with macular edema (RVO). This study aims to assess the effect of unintended delays in anti-VEGF treatment during the first wave of the COVID-19 pandemic. This retrospective case series identified patients receiving regularly scheduled anti-VEGF intravitreal injections based on current procedural terminology (CPT) code at two practices in Minnesota. Diagnoses were limited to nAMD, diabetic macular edema (DME), proliferative diabetic retinopathy, and RVO. Patients were divided into two groups based on whether they maintained or delayed their follow-up visit by more than two weeks beyond the recommended treatment interval during the COVID-19 lockdown. The 'COVID-19 lockdown' was defined as the period after March, 28th, 2020, when a lockdown was declared in Minnesota. We then compared the visual acuity and structural changes to the retina using ocular coherence tomography (OCT) to assess whether delayed treatment resulted in worse visual outcomes. A total of 167 eyes from 117 patients met criteria for inclusion in this study. In the delayed group, the average BCVA at the pre- and post-lockdown visits were 0.614 and 0.715 (logMAR) respectively (p = 0.007). Central subfield thickness (CST) increased from 341 to 447 in the DME delayed group (p = 0.03) while the CST increased from 301 to 314 (p = 0.4) in the nAMD delayed group. The results of this pilot study suggests that treatment delays may have a negative impact on the visual and anatomic outcomes of patients with nAMD and DME. Future studies with larger sample sizes are required for further investigation.

Comprehensive Characterization of COVID-19 Patients with Repeatedly Positive SARS-CoV-2 Tests Using a Large U.S. Electronic Health Record Database.

In the absence of genome sequencing, two positive molecular tests for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) separated by negative tests, prolonged time, and symptom resolution remain the best surrogate measure of possible reinfection. Using a large electronic health record database, we characterized clinical and testing data for 23 patients with repeatedly positive SARS-CoV-2 PCR test results ≥60 days apart, separated by ≥2 consecutive negative test results. The prevalence of chronic medical conditions, symptoms, and severe outcomes related to coronavirus disease 19 (COVID-19) illness were ascertained. The median age of patients was 64.5 years, 40% were Black, and 39% were female. A total of 83% smoked within the prior year, 61% were overweight/obese, 83% had immunocompromising conditions, and 96% had ≥2 comorbidities. The median interval between the two positive tests was 77 days. Among the 19 patients with 60 to 89 days between positive tests, 17 (89%) exhibited symptoms or clinical manifestations consistent with COVID-19 at the time of the second positive test and 14 (74%) were hospitalized at the second positive test. Of the four patients with ≥90 days between two positive tests (patient 2 [PT2], PT8, PT14, and PT19), two had mild or no symptoms at the second positive test and one, an immunocompromised patient, had a brief hospitalization at the first diagnosis, followed by intensive care unit (ICU) admission at the second diagnosis 3 months later. Our study demonstrated a high prevalence of compromised immune systems, comorbidities, obesity, and smoking among patients with repeatedly positive SARS-CoV-2 tests. Despite limitations, including a lack of semiquantitative estimates of viral load, these data may help prioritize suspected cases of reinfection for investigation and continued surveillance. IMPORTANCE The comprehensive characterization of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) testing and clinical data for patients with repeatedly positive SARS-CoV-2 tests can help prioritize suspected cases of reinfection for investigation in the absence of genome sequencing data and for continued surveillance of the potential long-term health consequences of SARS-CoV-2 infection.

COVID-19 SignSym: a fast adaptation of a general clinical NLP tool to identify and normalize COVID-19 signs and symptoms to OMOP common data model.

The COVID-19 pandemic swept across the world rapidly, infecting millions of people. An efficient tool that can accurately recognize important clinical concepts of COVID-19 from free text in electronic health records (EHRs) will be valuable to accelerate COVID-19 clinical research. To this end, this study aims at adapting the existing CLAMP natural language processing tool to quickly build COVID-19 SignSym, which can extract COVID-19 signs/symptoms and their 8 attributes (body location, severity, temporal expression, subject, condition, uncertainty, negation, and course) from clinical text. The extracted information is also mapped to standard concepts in the Observational Medical Outcomes Partnership common data model. A hybrid approach of combining deep learning-based models, curated lexicons, and pattern-based rules was applied to quickly build the COVID-19 SignSym from CLAMP, with optimized performance. Our extensive evaluation using 3 external sites with clinical notes of COVID-19 patients, as well as the online medical dialogues of COVID-19, shows COVID-19 SignSym can achieve high performance across data sources. The workflow used for this study can be generalized to other use cases, where existing clinical natural language processing tools need to be customized for specific information needs within a short time. COVID-19 SignSym is freely accessible to the research community as a downloadable package (https://clamp.uth.edu/covid/nlp.php) and has been used by 16 healthcare organizations to support clinical research of COVID-19.